-
Notifications
You must be signed in to change notification settings - Fork 227
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for prometheus expoter output plugin for fluentbit #840
Add support for prometheus expoter output plugin for fluentbit #840
Conversation
Signed-off-by: karan k <karan.k@oracle.com>
@@ -92,6 +92,8 @@ type OutputSpec struct { | |||
OpenSearch *output.OpenSearch `json:"opensearch,omitempty"` | |||
// OpenTelemetry defines OpenTelemetry Output configuration. | |||
OpenTelemetry *output.OpenTelemetry `json:"opentelemetry,omitempty"` | |||
// PrometheusExporter_types defines Prometheus exporter configuration to expose metrics from Fluent Bit. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We might need to add comments here and in the values.yaml
that it's recommended to add Prometheus exporter output only when the FluentBit is deployed as a single replica collector mode (A statefulset with 1 replica) because prometheus might scrape duplicated metrics if it's added to the default agent mode (FluentBit DaemonSet)
https://github.com/fluent/fluent-operator/blob/master/controllers/collector_controller.go
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Each Fluentbit pod will expose metrics related to itself, and it is probable for multiple pods to share identical metrics while exhibiting distinct values. Typically, customers require to scrape all metrics exposed from each pod.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it, you're using the Prometheus exporter output to collect metrics of all fluentbit pods. Then it make sense to me. I mean for other cases other than the metrics of fluentbit itself
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@benjaminhuo Let me know, if we are good
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@benjaminhuo Let me know, if we are good
yep, I'm ready to merge
What this PR does / why we need it:
Prometheus exporter output plugin is added for fluentbit so that fluentbit metrics can be exposed on the specific port such it can be scraped by the prometheus.
Which issue(s) this PR fixes:
Fixes #841
Does this PR introduced a user-facing change?
Additional documentation, usage docs, etc.: